Least squares type estimation of the transition density of a particular hidden Markov chain

Abstract: In this paper, we study the following model of hidden Markov chain: Yi = Xi + εi, i = 1, . . . , n + 1 with (Xi) a real-valued stationary Markov chain and (εi)1≤i≤n+1 a noise having a known distribution and independent of the sequence (Xi). We present an estimator of the transition density obtained by minimization of an original contrast that takes advantage of the regressive aspect of the problem. It is selected among a collection of projection estimators with a model selection method. The L-risk and its rate of convergence are evaluated for ordinary smooth noise and some simulations illustrate the method. We obtain uniform risk bounds over classes of Besov balls. In addition our estimation procedure requires no prior knowledge of the regularity of the true transition. Finally, our estimator permits to avoid the drawbacks of quotient estimators.